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I. INTRODUCTION 

A. PURPOSE 

This thesis examines the theoretical and practical aspects of simulation-based 
acquisition as a method for acquisition reform. It reviews the current acquisition process, 
defines and describes the current simulation-based acquisition environment and critically 
assesses risks, benefits, and feasibility of a simulation-based approach. It discusses key 
areas where a program manager might consider using simulation-based acquisition as a 
method for developing and acquiring weapon systems. The thesis also explores 
requirements for developing a simulation-based research laboratory, and recommends a 
high-level architecture for the environment. 
B: BACKGROUND 

The current Department of Defense (DoD) environment of shrinking budgets and 
acquisition reform motivated DoD to reduce system acquisition outlays through cost, 
schedule, and performance tradeoffs. Decisions made early in the design of a weapon 
system account for most of the life-cycle costs (Sanders, 1997). Recent advances in 
inexpensive computing technologies, computers, and simulations may provide acquisition 
program managers with tools for dramatically reducing cycle-time and program costs. 

Until recently, acquisition program managers primarily utilized computer models 
and simulations to improve product design decisions. In many instances, these decisions 
led to cost, schedule, and performance improvements. However, computer models and 


simulations were not usually viewed as integrated components for a product development 


and acquisition management. In many cases, managers were unaware of improvements to 
computer models and simulation tools that they could utilize to benefit their programs. 
Recent evidence suggests computer models and simulations may both reduce 
acquisition cycle-time up to 50% and lead to lower overall program costs (Sanders, 
1997). Assuming this trend continues, simulation-based acquisition in the future will 
combine state-of-the-art computer simulations and communications technology to create 
an integrated, focused approach to weapon systems acquisition leading to a significant 
reduction in the flyaway and life-cycle costs of a weapon system. 
Se THESIS OBJECTIVE 
This thesis examines the simulation-based acquisition process and evaluates how 
a research laboratory environment might benefit program managers who exploit this 
acquisition strategy. It explores the use of computer models and simulations both in the 
Government and commercial sector. Risks and benefits associated with the application of 
computer models and simulations to the acquisition process are also addressed. It 
suggests and discusses areas where a program manager might follow a simulation-based 
acquisition strategy. Finally, this thesis proposes a high-level architecture for designing a 
simulation-based laboratory where program managers can apply simulation-based 
acquisition methods to their programs. 
D. RESEARCH QUESTIONS 
The primary thesis research question 1s: 


1. How can a simulation-based laboratory assist acquisition managers? 


to 


Secondary research questions: 


1. What is simulation-based acquisition? 


NO 


What are the associated risks and benefits of a simulation-based acquisition 
Strategy? 


3. What key issues impact the use of simulation-based acquisition? 
E. SCOPE AND LIMITATIONS 

This thesis examines simulation-based acquisition from a management rather than 
a technical perspective. In doing so, it briefly overviews the current acquisition process 
and past attempts to reform and streamline the acquisition process. It then defines 
simulation-based acquisition, examines both DoD and civilian programs, and discusses 
risks and benefits of simulation-based acquisition. The thesis proposes a simulation- 
based acquisition laboratory architecture and suggests how it could benefit program 
managers. This thesis does not attempt to design specific models and simulations, or 
address unique programmatic situations. 
F. RESEARCH LITERATURE AND METHODOLOGY 

Research was conducted via personal, telephone, and electronic mail interviews 
with both program managers and modeling and simulation experts. Specifically, research 
data were gathered from official Government directives and policies, journals, previous 
theses, United States Code, DoD regulations, and personal interviews. Simulation-based 
acquisition information sources include the Defense Modeling and Simulation Office 


(DMSO), the Army Modeling and Simulation Office (AMSO), and the Army’s Tank- 


Automotive Command (TACOM). Commercial applications of modeling and simulation 
include Boeing and Lockheed Martin. 
G. ACRONYMS 
The appendix lists simulation-based acquisition acronyms used for this thesis. 
H. ORGANIZATION OF THESIS 

Chapter I contains the purpose, background, thesis objective, research questions, 
scope and limitations, and thesis organization. 

Chapter II addresses the current acquisition process. It reviews related policies 
and laws and provides an overview of the process. This chapter concludes by discussing 
recent reforms to help streamline the process. 

Chapter I] provides a background on modeling and simulation and defines 
simulation-based acquisition. The chapter examines several applications of modeling and 
simulation to both Defense and commercial acquisition ventures and identifies risks and 
benefits associated with simulation-based acquisition. 

Chapter IV overviews the laboratory environment and discusses implications for 
implementing a simulation-based acquisition laboratory. Finally this chapter suggests a 
high-level laboratory architecture. 

Chapter V gives conclusions, recommendations, and suggests areas for further 


research. 


Il. DEFENSE ACQUISITON 
A. INTRODUCTION 

The Defense acquisition process provides a method for acquisition managers to 
procure weapon and information systems needed by our Armed Forces to deter or win 
wars. Schmoll provides this insight into the Defense acquisition process: 

The Defense acquisition system is a single uniform system whereby all 

equipment, facilities, and services are planned, developed, acquired, 

maintained, and disposed of by the Department of Defense (DoD). The 
system includes policies and practices that govern acquisition, identifying 

and prioritizing resource requirements, directing and controlling the 

process, contracting, and reporting to Congress. (Schmoll, 1996) 

Defense acquisition provides warfighters with equipment necessary to fight and 
win on tomorrow’s battlefields. As such, the process requires close coordination between 
DoD, users, acquisition managers, Congress, and the commercial sector. The Defense 
acquisition process translates a need or requirement into a tangible, usable, and useful 
system the warfighter can employ on the battlefield. Schmoll defines successful Defense 
acquisition as: 

A program that places a capable and supportable system in the hands of a 

user when and where it is needed, and does so within the bounds of 

affordability (Schmoll, 1996). 


The remainder of this chapter focuses primarily on the management of Major Defense 


Acquisition Programs (MDAPs). 


B. AUTHORITY FOR DEFENSE ACQUISITION 

Defense acquisition authority and guidelines are found in a number of sources 
including Federal laws, Office of Management and Budget circulars, Federal regulations, 
and agency supplements to Federal regulations. 

Ik Federal Laws 

Laws set forth by Congress provide the foundation for successful acquisition of 
materials and services that support Defense requirements. Key laws include Title 10 of 
the United States Code, DoD Procurement Reform Act (1985), DoD Reorganization Act 
of 1986 (Goldwater-Nichols Act), and the Federal Acquisition Streamlining Act (FASA) 
of 1994. 

ay Office of Management and Budget Circular A-109 

Office of Management and Budget (OMB) Circular A-109 establishes overarching 
acquisition policy for all Federal organizations. It directs all Federal agencies to establish 
clear management authority by designating a program manager (PM) for each acquisition. 
Further, it requires competitive exploration of alternative systems. 

3. Federal Acquisition Regulation 

The Federal Acquisition Regulation (FAR) governs the acquisition of supplies and 
services by all Federal organizations. This regulation broadly defines acquisition policy 
while allowing individual organizations to tailor the guidance to their specific needs. It 
provides a procurement framework based on sound business practices to assist the PM in 


acquisition management. 


4. Agency Supplements 
Federal agency supplements tailor the FAR by further defining how agencies 
accomplish product and service acquisition. Agency Supplements to the FAR provide 
employees with specific guidance for conducting acquisition management. For example, 
the DoD 3000 provides key policy and defines procedural language for Defense 
acquisition. It applies to all Defense acquisitions but, in particular, addresses MDAP 
management. Within this series, two key documents that guide Defense acquisition are 
DoD Directive 5000.1 and DoD Regulation 5000.2-R. 
a. DoD Directive 5000.1 
DoD Directive 5000.1 provides guidelines for acquiring all materials and 
services. It establishes broad policy and underlying principles that provide for the 
integrated management of all Defense procurements. DoD Directive 5000.1 highlights 
three principles critical to the success of an acquisition program: translating operational 
needs into stable and affordable programs, acquiring quality products, and Saad for 
efficiency (Schmoll, 1996). 
b. DoD Regulation 5000.2-R 
DoD Regulation 5000.2-R mandates policies and procedures for the 
acquisition of MDAPs. The regulation provides procedural guidance for six key 
acquisition management areas (Schmoll, 1996). 


e Acquisition Management Process: Provides a general model for MDAP 
management. 


e Program Definition: Provides guidance for translating broad needs into well- 
defined performance specifications. 


e Program Structure: Identifies elements such as objectives and thresholds, 
contract type, and Cost As an Independent Variable (CAIV) necessary to 
configure a successful program. 

e Program Design: Establishes the basis for a structured and disciplined 

approach to MDAP life-cycle management through the use of Integrated Product 

and Process Development (IPPD) and the systems engineering process. 

e Program Assessment and Decision Reviews: Establishes mandatory policies 

and procedures for conducting periodic assessments and milestone decision 

reviews. 

e Periodic Reporting: Describes mandatory reports, and in some cases provides 

report formats required by oversight agencies such as Acquisition Executives 

(AEs) and Congress. 

C. THE DEFENSE ACQUISITION PROCESS 

Defense Acquisition is event-driven. It follows a structured, yet tailorable, 
approach to developing, fielding, and disposing weapon systems. 

1. Overview of Acquisition-Based Methodologies 

Program managers may utilize several methods for managing an acquisition 
program. These include: traditional, grand design, incremental, and evolutionary 
methodologies. Each acquisition methodology is briefly described below. 

Traditional. The Traditional methodology serves as DoD’s baseline approach for 
MDAP acquisition. It follows a linear, phased approach for managing programs. This 
methodology is generally used for high-risk MDAP acquisition. This methodology 
typically takes up to 15 years to acquire a Weapon system. 

Grand Design. The Grand Design methodology 1s utilized when requirements are 


well-defined and additional design improvements are not needed. All requirements for 


system acquisition, development, and deployment are conducted in one step. 


Incremental. PMs utilize the Incremental methodology when requirements are 
well-defined, but other acquisition processes, such as funding, are less certain. PMs field 
the system in a series of increments, or blocks, with increasing functionality over time 
rather than fielding a fully functional system at a single point in time. This methodology 
is characterized by a “build-a-little, test-a-little” approach that delivers a functional subset 
of the final capability at the initial and subsequent blocks. 

Evolutionary. The evolutionary methodology is characterized by the design, 
development, and deployment of a capability using currently-available technologies, but 
also includes provisions for the addition of future capabilities as the requirements and 
technologies mature. This methodology is often utilized when the requirements are not 
fully defined, but evolve over time as the system is designed, developed, and prototyped. 
Often this approach is best suited to situations where detailed system or operational 
requirements are difficult to articulate. Advanced Technology Demonstrations (ATDs) 
can employ this approach. 

i Baseline Methodology 

This section focuses on the baseline methodology by discussing the components 
associated with it. Note that the components are also common to other acquisition 
methodologies but may not be completed in the same sequence. Phases correspond to 
time periods during which activities occur that contribute to a common outcome. 
Milestones act as decision points, allowing an approval authority to make an informed 
decision whether to move into the next phase. Figure | depicts the elements that are 


considered at each milestone. These phases and milestones may be repeated to create a 
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Figure |. Acquisition Phases and Milestones (DoD Acquisition Deskbook, 1998). 





specific acquisition process structure. Figure 2 depicts the baseline acquisition 


methodology. This methodology consists of four phases and milestones. 
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Figure 2. The Baseline Acquisition Methodology (DoD Acquisition Deskbook, 1998). 


a. Pre-phase 0 

Pre-phase 0 activities involve the determination of a mission need as 
depicted in Figure 2. Pre-phase 0 begins with a Mission Area Analysis (MAA) which is 
an examination of current and projected DoD capabilities, future threats, changes in 


policy, as well as an analysis of technological advancements in a particular category. The 
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MAA, conducted by organizations such as the US Army Training and Doctrine 
Command (TRADOC), may examine areas such as tank warfare. If the MAA identifies a 
deficiency or weakness in this category, a non-material solution is sought. Changes to 
doctrine or tactics may be all that is required for the non-material solution. If an 
acceptable non-material solution is not available, a mission need statement (MNS) is 
developed. The MNS document formally states the requirement for a material solution. 
Once the MNS is completed it must be validated. For a MNS that will lead to a MDAP, 
the Joint Requirements Oversight Council (JROC) is the validation authority. For non- 
MDAP MNS, validation authority may be delegated as low as Commanders-in-chief 
(CINCs) of unified commands. Upon validation the MNS 1s sent to the Under Secretary 
of Defense (Acquisition and Technology) (USD(A&T)) for a Milestone O decision. 

b. Milestone 0 

Milestone O consists of the approval by the USD(A&T) or other 
appropriate Milestone Decision Authority (MDA) to proceed into Phase O and conduct 
concept studies. The MDA specifies a minimum set of alternatives to be studied, the lead 
organization, and the exit criteria for Phase 0. 

i Phase 0 

Phase 0 consists of concept exploration. Alternative concepts are studied 
to determine their feasibility. Phase 0 is characterized by competitive and parallel studies 
that provide the data necessary to assess the merits of each concept. Phase 0 usually lasts 


about | to 2 years. (Schmoll, 1996) 


d. Milestone I 

Approval! to begin a new acquisition program occurs at Milestone I. The 
MDA approves the acquisition strategy and baseline that allows for the entry into Phase I. 
Other objectives, such as CAIV and exit criteria for Phase I, are also defined. (Schmoll, 
ODay) 

é. Phase I 

Phase I is Program Definition and Risk Reduction (PDRR). It focuses on 
mitigating risk through various methods such as early prototyping and cost-performance 
trade-offs. The PM identifies cost-drivers, interoperability issues, and alternatives to the 
acquisition strategy. This phase can last upwards of 5 years, especially if prototyping is 
conducted. (Schmoll, 1996) 

f. Milestone II 

The MDA gives approval to enter the engineering and manufacturing 
development phase at this milestone. The acquisition strategy and baseline are revisited 
and approved, exit criteria established for Phase Hf and quantities identified for low rate 
initial production (LRIP). (Schmoll 1996) 

g. Phase IT 

Engineering and Manufacturing Development (EMD), or Phase II, ensures 
the designed system is ready for manufacturing. To accomplish this, a number of 
certifications must occur including a design review and production process validation. 


This phase also encompasses testing in the form of Developmental Test and Evaluation 


(DT&E). The system is ready to enter into Phase III when the early prototype or final 
design is approved and the manufacturing processes are validated. (Schmoll, 1996) 

h. Milestone II 

The production decision 1s made at Milestone II[. The MDA reviews and 
approves the acquisition strategy and baseline. The decision at this milestone allows the 
program to enter full-rate production. 

I. Phase III 

Phase III, the longest of all phases, encompasses a variety of activities. Not 
only are the systems produced during this phase, but all fielding, operational support, and 
disposal activities are covered as well. As the system is produced, additional testing is 
conducted primarily in the form of Operational Test and Evaluation (OT&E) which 
allows an independent assessment of the system’s effectiveness and suitability. This 
testing is generally conducted in a live setting. Once the system is fielded, issues such as 
modifications to sub-components are dealt with on a case-by-case basis. Additionally, as 
the system begins to wear out or a replacement is fielded, demilitarization and disposal of 
the system is overseen by the PM. This helps ensure that environmental, safety, and 
security issues are considered and applied. 
C; ACQUISITION REFORM 

Over the last 50 years the process of developing and acquiring weapon systems 

has become complex and inefficient often characterized by cost overruns and fifteen-year 


procurement cycles. Dr. Kaminski, the USD (A&T) in the 1990s, noted that: 
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New national security challenges require DoD to design a more flexible, 

agile, and timely acquisition system capable of meeting unpredictable 

threats. This means that the DoD acquisition system must improve it’s 

support to the war fighter by reducing the acquisition cycle time and 

leveraging the latest available technologies, particularly information 

technology (Kaminski, 1997). 

Throughout the past 50 years, numerous attempts were made to change the way 
DoD procured weapon systems. It was not until the 1990s that a number of initiatives 
were successful in reducing the cost and cycle-time associated with acquisition. Schmoll 
notes that unlike previous decades, the 1990s: 

Can be characterized as an era with declining Defense budgets, smaller 

workforces, and significant changes 1n threat to national security (Schmoll, 

Sieve 
These factors, combined with the visionary leadership by persons such as the then 
Secretary of Defense Dr. William Perry and Dr. Kaminski, as well as the adoption of key 
legislation, created an environment that allowed acquisition reform to take place. 

1. Legislation 

Section 800 of Public Law 101-510 provides the basis for acquisition reform in 
the 1990s. This Act established a framework that included the following goals: 
streamline the Defense acquisition process, eliminate unnecessary acquisition laws, 
ensure the financial integrity of defense procurement programs, and protect the best 
interests of DoD. (Schmoll, 1996) 

Two additional legislative documents provide more concrete direction for 


implementing changes to Defense acquisition. The Federal Acquisition Streamlining Act 


of 1994 and the National Defense Authorization Act of 1996 outlined changes to the 


14 


acquisition process that assisted in making acquisition reform a reality. These changes 
attempt to streamline the acquisition process by reducing overall program complexity, 
cost, and cycle-time. 
ja Initiatives 
DoD has undertaken many initiatives since these Acts took effect. Some key 
initiatives are described below: 
a. Electronic Commerce / Electronic Data Interchange 
The Electronic Commerce / Electronic Data Interchange (EC/EDI) 
initiative facilitates the conduct of business transactions and the exchange of information 
electronically. DoD and commercial entities can complete the contracting, payment, and 
data exchange processes through the use of electronic media and in some cases via the 
internet. EC/EDI eliminates massive amounts of paperwork, facilitates instant 
communications, and automates bulk transaction processing. 


b. Integrated Product and Process Development 


The Integrated Product and Process Development (IPPD) initiative 
attempts to remove barriers that existed from both an inter- and intra-Program 
Management Office (PMO) perspective. Previously, PMOs operated in a stove-piped 
fashion, with each functional discipline, such as finance or engineering, essentially 
Operating on their own. This led to inefficiencies in communication and in 
accomplishment of tasks due to the compartmentalized nature of the PMO. IPPD creates 
an environment for individuals of specific functional disciplines to meet together to 


collectively plan and execute a particular task. In doing so, cross-functional problems are 
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resolved early-on and information flows both horizontally and vertically through the 
PMO. Additionally this initiative brings DoD and the contractor together to work side- 
by-side in the development of the product. 

c. Single Process Initiative 

This initiative attempts to eliminate or consolidate redundant 
specifications that a contractor must utilize to satiate the requirements from different 
Services or commercial sectors for the same product. By eliminating or reducing 
numerous variations on a particular specification or standard, the costs of maintaining, 
overseeing, and producing products under multiple specifications or standards is reduced. 

d. Simulation Test and Evaluation Process Improvements 

A number of activities are directed at reducing the cost and duration of 
conducting operational testing on emerging systems. The Simulation Test and Evaluation 
Process (STEP) improvements are aimed at bringing the operational test community into 
a program as early as possible. Working under the concept of IPPD, these improvements 
consolidate developmental and operational testing and inject more computer modeling 
and simulation into the testing program. Through the consolidation of these tests, 
reductions in cycle-time and costs are attained. 
Dy. SUMMARY 

This chapter examined and discussed the Defense acquisition process. It provided 

an overview of the process and some recent initiatives aimed at streamlining an often 
complex and inefficient process. It provides a foundation for understanding the 


complexity associated with the acquisition process and the need for reform in a time 
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when defense budgets are declining and the geopolitical environment has significantly 
changed. Chapter IV examines how simulation-based acquisition can answer the call for 
acquisition reform and provide the PM with a method for decreasing costs, cycle-time, 


and improving performance. 


1 





Il. SIMULATION-BASED ACQUISITION 

A. INTRODUCTION 

DoD supports applying modeling and simulation across a wide range of 
disciplines. Historically, DoD primarily utilized modeling and simulation to support 
training and operational assessment. Recently, the DoD acquisition community focused 
on utilizing computer models and simulations for developing, designing, and 
manufacturing weapon systems. This concept, now referred to as Simulation-Based 
Acquisition (SBA), represents an effort to fully integrate modeling and simulation 
technology across all acquisition functions and phases. 
B. REGULATIONS AND GUIDANCE 

The DoD 5000 series emphasizes the use of modeling and simulation technology. 
DoD Directive 5000.1 states: 

Models and simulations shall be used to reduce time, resources, and risks 

of the acquisition process and to increase the quality of systems being 

acquired. Representations of proposed systems (virtual prototypes) shall 

be embedded in realistic, synthetic environments to support the various 

phases of the acquisition process, from requirements determination and 

initial concept exploration to the manufacturing and testing of new 

systems, and related training (DoD Directive 5000.1, 1996). 
DoD Regulation 5000.2-R mandates the use of modeling and simulation when 
appropriate: 

Accredited modeling and simulation shall be applied, as appropriate, 

throughout the system life-cycle in support of various acquisition 

activities: requirements definition, program management, design and 

engineering, efficient test planning, result prediction, and to supplement 

actual test and evaluation, manufacturing, and logistics support. PM's 


shall integrate the use of modeling and simulation within the program 
planning activities, plan for life-cycle application, support, and reuse 
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models and simulations, and integrate modeling and simulation across the 

functional disciplines (DoD Regulation 5000.2-R, 1998). 

DoD Directive 5000.59-P provides a broad framework for applying computer models and 
simulations across the DoD acquisition process. It emphasizes the following key point: 

Defense modeling and simulation will provide readily available, 

operationally valid environments...to support technology assessment, 

system upgrade, prototype and full-scale development, and force structure 

(DoD Directive 5000.59-P, 1995). 

This Directive further states: 

Common use of these environments will promote a closer interaction 

between operations and acquisition communities... and modeling and 

simulation environments will be constructed from affordable, reusable, 
components interoperating through an open systems architecture (DoD 

Directive 5000.59-P, 1995). 

Finally, DoD 5000.59 delineates responsibilities for managing acquisition-based 
models and simulations and establishes the Defense Modeling and Simulation Office 
(DMSO) as the key integrator of models and simulations for the research, development, 
and acquisition domains (DoD Directive 5000-59, 1994). The DoD 5000 series points 
out that use of modeling and simulation technologies may greatly assist acquisition 
professionals in developing and delivering less expensive, yet more effective, weapon 
systems to the warfighter. Each regulation cited above recognizes, in its own way, the 
positive benefits and value-added of modeling and simulation applied to SBA. 


High-level DoD support for using modeling and simulation in SBA comes from 


officials such Dr. Jacques S. Gansler, the USD(A&T). In comments to the 1998 National 


20 


Defense Industrial Association (NDIA) Simulation-Based Acquisition Workshop, Dr. 
Gansler states: 

I believe the payoff from acquisition modeling and simulation is now, not 

far off into the future...As more and more of our acquisition workforce 

‘buy in’ to the notion that modeling and simulation can pay big dividends 

— both in terms of reduced cycle time and reduced costs — the barriers to 

this impressive technology will collapse (Gansler, 1998). 

He further observed that the Defense acquisition community has not yet fully recognized 
the potential benefits of modeling and simulation: 

..the potential here is enormous — especially in linking performance and 

costs together, going through the details of design, production and support 

(Gansler, 1998). 

Other key individuals who, based on public statements, favor the use of SBA 
include Dr. Patricia Sanders, Director, Defense Test, Systems Engineering and 
Evaluation, and Dr. Herbert Fallin, Director, Assessment and Evaluation, Office of the 
Assistant Secretary of the Army for Research and Development. 

Cc: MODELS AND SIMULATIONS 

The Defense Systems Management College (DSMC) defines a model as a 
physical, mathematical, and logical representation of a system, entity, phenomenon, or 
process (Mercer, 1994). One can readily understand the category of physical models. 
For instance, a child who builds a ‘model’ airplane creates a physical representation of 
the actual airplane. In industry, however, system developers often create very complex 
computer models of real world systems and processes. This gives them the capability, for 


example, to fully design a product using a computer to depict its components from two or 
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three-dimensional viewpoints. The DoD acquisition community hopes to leverage these 
same methods to design and develop future military weapon systems. 

DSMC defines simulation as a method for implementing a model over time; and a 
technique for testing, analyzing, or training in which real-world systems are used, or 
where real world and conceptual systems are reproduced by a model (Mercer, 1994). 
Computer simulations allow engineers and analysts to examine system behavior in much 
less time than it would actually take to evaluate a system prototype in a live environment. 
Simulations also save valuable resources of time, aanoonee and money. For example, 
through the use of computer simulation, a pilot can fly an aircraft prototype through near- 
realistic, albeit, stmulated conditions without requiring development of an actual aircraft. 
Many of today’s video games illustrate this type of simulation. 

The term, modeling and simulation (M&S), refers to the use of models, including 
emulators, prototypes, simulators, and simulations to generate data as a basis for making 
managerial or technical decisions (DoD Acquisition Deskbook, 1998). Throughout this 
thesis, M&S refers to the collective use of models and simulations according to this 
definition. Within this context, it is important that program managers understand that the 
M&S used for making decisions are only as good as the algorithms, framework, and 


inputs used to design and run the models and simulations. 


to 
to 


Models and simulations may be categorized into three broad classes as depicted in 
Figure 3. Engineers and analysts generally develop and use computer-based constructive 


models and simulations. High-level simulations generate data on large-scale battles or 


DESCRIPTION CLASSES EXAMPLES 


Wargames models, and eEngineering models 
analytical tools. CONSTRUCTIVE “Cost models 


eSupport models 


hoe eAircraft simulators 
Systems simulated both physically and 5 ; 
by computer. Troops in simulators VIRTUAL pel Pe 


fight on synthetic battlefields. “Trainers 


Operations with live forces and real | Live *REFORGER 
equipment in the field. LIVE eNational Training Center 


eInstrumented prototypes 





Figure 3. Classes of Models and Simulations (Mercer, 1994). 


campaigns while lower-level models render specific design or cost data. The acquisition 
community currently uses constructive models for weapon system development. 

Virtual simulations merge an actual system, subsystem, or virtual prototype with 
humans in a synthetic environment. This includes, for example, “man-in-the-loop” 
operating an aircraft or tank simulator at a stationary location in a computer-generated 
environment. 

Live models or simulations include training exercises where participants perform 
actions and operations short of combat. Soldiers operate assigned equipment under 
realistic conditions. Examples of live simulations range from Return of Forces to Europe 


(REFORGER), to Louisiana Maneuvers, to simple, platoon-level training exercises in 
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local training areas. Currently, operational testing mostly relies on live simulation. The 
purpose of all three types of models and simulations is to gather data on the performance 
of a system or process in peacetime, so leaders or managers can more thoroughly 
understand how the system or process will perform under combat conditions. 

One may also categorize models and simulations based on a hierarchical view of 
military operations. Models and simulations of different levels of military operations 
vary greatly in their ability to show detail. Mercer describes four levels of military 
models and simulations: theater/campaign, mission/battle, engagement, and engineering 
(see Figure 4 below). 

At the theater level, models and simulations assess joint or combined operations. 
They generally give ““big-picture” outcomes of interactions between large forces while 
varying equipment and capabilities. At the mission level, the focus narrows to a force 
package, manned and equipped to accomplish a specific mission. At lower engagement 
levels, simulations support studies comparing one weapon system against anotl For 
example, simulation data may be generated for an engagement between a United States 
M-1 and a Russian T-81 tanks. Engineering-level models depict the greatest level of 
detail. Among other things, engineering models examine design, cost, human-machine 
interface, and manufacturing options. Engineering-level models assist managers in 


evaluating specific performance criteria of an individual component or sub-component. 
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Figure 4. Hierarchy of Models and Simulations (Mercer, 1994). 


The previous discussion presented an overview of model classes and simulations 


of different levels of military operations. The development of robust internet browsers 


and improved communications opened new opportunities for creating hybrid simulation 


environments. In the future, program managers may use hybrid simulations to create 


unique simulation environments to support acquisition processes. Mercer identifies 


several hybrids consisting of multiple classes, hardware and software combinations, and 
simulation environments (Mercer, 1994): 


e Distributive Simulation. This concept represents the ability to link different 
types of simulators across geographically dispersed locations. 


e Stimulation. Computer models or simulations provide data that feeds directly 
to an actual system. This data provides simulated threats or other phenomena for 
testing a weapon system. 

e Hardware/Software-In-The-Loop (HW/SWIL). HW/SWIL combines system 
hardware, multiple classes of models or simulations, and external stimuli. This 
environment demonstrates a systems capacity to operate within an environment 
that simulates actual conditions. The Guided Weapons Facility (GWEF) at Eglin 
Air Force Base is one example HW/SWIL. 


D. APPLICATION OF MODELS AND SIMUALTIONS TO THE 
ACQUISITION PROCESS 


M&S can potentially support all phases of the acquisition process. Prior to Phase 
0, engineering models provide insight into performance aspects of new system designs, 
while campaign or mission level simulations can assist in evaluating the consequences of 
implementing the new technology prior to production. During Phase 0, engineering-level 
models and simulations may assist managers in conducting cost and operational 
effectiveness studies to support the analysis of alternatives. Mission and theater-level 
simulations may provide data for evaluating system effectiveness and performance. 
Program managers may use human-interactive simulations to examine and revise tactics 
and doctrine, resulting from the application of a new technology in system design 
(Mercer, 1994). Also during Phase 0, program managers prepare the Simulation Support 


Plan (SSP). The SSP outlines how program managers plan to utilize M&S throughout 
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the program. The SSP also assists program managers with planning and implementing 
M&sS technologies throughout the life-cycle of the weapon system. 

During Phase I, program managers may use models and simulations to identify 
and solve technical problems early in the acquisition process. Design and production 
improvements may be achieved through the use of Computer-Aided Design (CAD) and 
Computer-Aided Manufacturing (CAM) linked to constructive and virtual simulations of 
production facilities. Simulations of production facilities may improve planning 
production processes and scheduling production. Cost models that incorporate 
engineering life-cycle costs may help evaluate alternatives presented in the analysis of 
alternatives. Simulations may provide data for early performance assessments during this 
phase instead of requiring contractors to produce data using prototypes (Mercer, 1994). 

In Phase II, engineering-level models may be used to assess designs, design 
changes, capability trade-offs, test planning and support, subsystem and system 
performance, and verification of specification compliance. Computer models and 
simulations also support developmental test and evaluation, operational test and 
evaluation, as well as production and fielding (Mercer, 1994). Program managers may 
use HW/SWIL for pretest planning, execution, and analysis. This will help identify 
hardware problems prior to conducting live tests. Models and simulations seem well 
suited to support logistical and human support analyses by providing data on system 
reliability, availability, and maintainability. Engineering model data on LRIP hardware 


further support cost analysis. A combination of engineering, engagement, mission, and 
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campaign simulations may assist live simulations of both developmental and operational 
test and evaluation. 

In Phase IJ, M&S supports testing, operator training, development of tactics, 
techniques, and procedures, and reevaluating design changes. Live simulations support 
OT&E ensuring the actual system meets required specifications and performance 
parameters. Virtual simulations assist in training system operators and maintainers and 
also contribute to mission rehearsal. Engagement, mission, and campaign-level models 
and simulations may assist in developing or refining tactics and doctrine. Finally, 
engineering-level models and simulations may provide insights into production and 
performance prior to the PM implementing design changes. The next sections discuss 
how acquisition managers applied M&S to the acquisition process in the past and 
implications of using M&S in the future. 

E. HISTORICAL APPLICATION OF MODELS AND SIMULATIONS 

In the past, acquisition managers applied M&S to the acquisition process with 
varying success. Figure 5 depicts how program managers utilized M&S prior to SBA. 
According to Karangelen (1997), program managers viewed M&S as a collection of 
disconnected applications that provided limited decision-making support to very specific 
functional domains. M&S was not integrated into the acquisition process, causing 
significant delays when using them to evaluate acquisition milestone decisions. Poor 
communications connectivity, and the lack of a functionally integrated environment 


added to this delay. Although program managers often attempted to apply M&S 
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Figure 5. M&S and the Acquisition Process (Karangelen, 1997). 


throughout the entire acquisition process, they primarily invested in M&S that supported 
training and operational analysis issues. 

From early results, acquisition clearly benefited from modeling and simulation 
applications. System engineers recognized the value of using CAD tools to design 
systems in computer environments. These tools made it possible to easily and quickly 
generate two or three-dimensional designs to help contractors and warfighters visualize 
the system. CAD allowed designers to design, change, and evaluate drawings in a 
fraction of the time it normally took draftsman to draw blueprints by hand. This 
modeling environment conserves substantial resources required previously to design a 
system (Gansler, 1998). CAD also provided insights into problems caused by 
interference. Interference refers to instances where system parts or components 


physically occupy the same location causing manufacturing problems. Through CAD 
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methods, engineers can check for interference prior to building the system saving 
program managers time and resources. 

Simulations, combined with cost models, support analysis of production processes 
and schedules, performance tradeoffs, and logistical and maintenance requirements. An 
October 1996 study, commissioned by Dr. Patricia Sanders, identified a number of 
benefits from applying M&S to key acquisition areas: concept development, engineering 
design and analysis, test and evaluation, manufacturing, and deployment and support 
processes. The report highlighted where M&S benefited acquisition programs in the past 
(Sanders, 1996): 


e Concept Development. DoD cancelled proposed development of the 
Wide-Area, Ant-Armor Munition (WAAM) and the JP-233 Runway- 
Attack Munition by identifying poor lethality potential. This resulted in a 
cost avoidance of $84M that may have been otherwise used to develop 
systems that did not meet user requirements or expectations. 


e Engineering Design and Analysis. Using the CAD environment, the 
Tank Automotive Research, Development, and Engineering Center 
(TARDEC) designed a new, low-silhouette tank prototype using 14 
engineers in 16 months, as opposed to the normal 55 engineers in three 
years. In another example, Sikorski Aircraft employed 38 draftsmen and 6 
months to produce working drawings of the CH-53E Super Stallion’s 
outside contours, while the M&S environment employed in the Comanche 
program required only one engineer to do the same task in 1 month. 


e Test and Evaluation. Program managers accomplished bridge 
durability testing by conducting 3000 actual crossings at a cost of about 
$325K over a 12-month time frame. The combined use of constructive 
and real [live] simulations for this testing yielded cost savings of 
approximately $110K and a schedule reduction of approximately two 
months. The Moving Target Simulator (MTS) assesses the ability of tanks 
to engage moving targets in a simulated environment. It saves the Army 
approximately $1.5M annually compared with running live field tests. 
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e Manufacturing. The Joint Strike Fighter (JSF) program reconfigured a 

component, which originally contained 250 parts, to just 25 parts. This 

made it much easier to manufacture. The Army’s use of the Flexible 

Computer-Aided Integrated Manufacturing (FCIM) program in electronics 

manufacturing led to a 66% reduction in cycle-time, $3M in cost savings, 

and $3.8M in cost avoidance. 

e Deployment and Support. Electronic provisioning by Northrup 

reduced release time from six months to 60 minutes. The AH-64D 

Longbow Program utilizes a simulation to provide for high complexity 

maintenance training in a simulated environment, greatly contributing to 

the efficiency of maintenance personnel in the field. 

Although these examples illustrate benefits of using M&S in Defense acquisition, 
problems still exist with M&S that impact program management. Notable problems 
include: 

e Specialized M&S tools offering limited reuse potential; 

e Lack of interoperability between models and simulations; 

e Nocentralized M&S database; 

e Unavailability of inexpensive, sophisticated, and high performing computing 

technologies; 

e Inaccessible high-speed distributive communications environment. 

From a management perspective, these M&S shortfalls inhibit development of a mature 
and robust SBA paradigm. 

The lack of a focused Integrated Product and Process Development (IPPD) 


environment limits sharing information across functional areas. As functional experts 


developed unique M&S applications to support their needs, they neglected to plan for 
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M&s connectivity with other systems or reuse. The reuse issue applies to both intra- 
program and inter-program functional domains. 

A lack of M&S and computer science training and communication through out the 
acquisition community made it difficult for program managers to capitalize on the 
potential benefits of applying M&S across all functional domains and phases of the 
acquisition life-cycle. The acquisition community simply did not understand how the 
analysis, testing, and M&S communities used live experiments or constructive 
simulations such as JANUS, CASTFOREM, VIC, or Eagle to support force development 
studies. Therefore, the acquisition community failed to fully comprehend the benefits of 
applying constructive and virtual models and simulations to the acquisition process. 

The belief by some key leaders within the acquisition community that M&S could 
not produce accurate and reliable data also contributed to the problem (Sanders, 1996). 
Consistent and practical guidelines outlining methods for validating and verifying models 
and simulations for acquisition did not exist. For these reasons, organizations such as 
OT&E resisted the use of constructive and virtual M&S relying, instead on expensive, 
time and resource intensive live exercises. 

Technology constraints also limited the use of M&S for acquisition. 
Collaborative, constructive, and virtual M&S require fast computers and highly 
sophisticated software applications. Until recently, no DoD M&S architecture existed for 
development of an integrated model and simulation environment enabling systems to 


communicate with each other. The lack of high-performing, inexpensive computers also 


made it difficult to integrate distributed M&S. For these reasons, managers generally 
stove-piped models and simulations within a narrow application domain. 

Finally, an immature information technology environment also inhibited 
application of models and simulations. A lack of reliable, economical, high-speed data 
exchange and communication systems meant program managers could not access large 
bandwidths needed for distributed communications. This restricted the use of M&S to 
specific locations making it necessary for acquisition team members to travel to the M&S 
testing site. 
|i SIMULATION-BASED ACQUISITION 

Over time, the M&S and acquisition communities worked together to overcome 
the shortcomings of applying M&S to the acquisition process. This led to a new 
paradigm -- Simulation-Based Acquisition. The American Defense Preparedness 
Association defines SBA as: 

An advanced engineering environment that includes formal methods, 

tools, and a common standard. It provides the means for executing an 

extensible, tailorable, and repeatable process that results in creation of 

reusable design repositories and re-engineerable products. It is a 

comprehensive integrated systems engineering environment coupled with 

an iterative acquisition process and an evolved culture (Portman, 1996). 


Portman (1996) describes SBA as: 


e An advanced engineering environment which utilizes formal methods, tools, 
and representations. 


e A tailorable, iterative acquisition process. 
e An Integrated Process and Product Development (IPPD) environment that 


includes refined roles and responsibilities of key acquisition personnel and 
agencies. 
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The SBA paradigm relies on all types of M&S, however, it focuses mainly on the use of 
constructive and virtual models and simulations to reduce the level of resources 


previously required to conduct live M&S. Fully leveraging this paradigm in the future 
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Figure 6. SBA Elements (Karangelen, 1997). 





will mean program managers must address technological barriers that precluded the use 
of M&S in the past. Figure 6 depicts key M&S components. 

First, the acquisition community must integrate multiple M&S applications. . That 
is, computer models and simulations must seamlessly support all acquisition phases and 
disciplines. Computer models and simulations must share common databases and 
communication protocols. Figure 7 depicts this environment. One promising new M&S 
initiative that will benefit SBA is specification of a High-Level Architecture (HLA) 
environment. Once mature, HLA will provide a common, interoperable framework for 


models and simulations (DMSO, 1998). 
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Figure 7. Advanced Engineering Environment (Frost, 1998). 


Second, the acquisition and M&S communities must work together to articulate 
the verification, validation, and accreditation (VV&A) process for SBA that establish 
M&sS credibility within the acquisition community. Verification determines that a model 
accurately represents the developer's conceptual description and _ specifications. 
Validation determines the degree to which a model accurately represents the real world. 
Accreditation certifies a model or simulation for a specific purpose. The responsibility 
for reviewing new models and simulations to ensure they produce reliable results belongs 
jointly to M&S developers and users. VV&A support for the acquisition community 
comes mainly from two agencies. The Training and Doctrine Command Analysis Center 
(TRAC) serves as the Army’s VV&A subject matter expert (Army Regulation 5-11, 
1997). As such, TRAC helps bridge technical gaps for developing reliable and credible 
models and simulations for SBA. DoD designated DMSO as the central management 


office for all Defense M&S (DoD Instruction 5000.61, 1996). DMSO provides M&S 
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expertise On a wide range of technical and technological issues and also serves as the 
DoD Modeling and Simulation Resource Repository (MSRR). 

Third, the application of M&S to SBA requires reengineering the current 
acquisition processes. The “spiral” systems engineering method seems well-suited for 
integrating M&S into SBA. Applied over time, the spiral systems engineering 
subprocesses of concept identification, decomposition, synthesis, and integration help 
ensure development of acceptable products. The systems engineering process addresses 
the repeated questions of why, what, and how of the users, program management office 
representatives, and contractors as depicted in Figure 6. This allows system engineers 
and program mangers to continually re-evaluate mission needs, requirements, design 
trade-offs, improvements, and alternatives. SBA exploits the systems engineering 
process by compressing each iteration of the spiral through the use of constructive and 
virtual M&S. 

For example, the Army recently employed this process. Figure 8 depicts a recent 
application of this process. It shows an evolutionary cycle involving continuous soldier 
(user) feedback linked to system development through program management and 
contractors. The Army tested this process through Army Warfighting Experiments 
(AWEs). In these experiments, program managers provided developmental technology 
and products for soldiers to use in training exercises. Live exercises supported by virtual 
and constructive simulations provided insights into the impacts of new information 


technologies on doctrine, tactics, techniques, and procedures. 
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Additionally, contractors, who attended the exercises, made on-the-spot changes 
to the equipment or suggested alternate employment methods to further improve results 
through this approach. This approach, referred to as “design-test-design,’ has been 
described by LTG Kern, the Military Deputy to the Army Acquisition Executive, as the 
spiral development process. He envisioned this process as one of the keys to SBA (Kern, 
LOOS): 

The maturing of a simulation-based environment for acquisition depends upon a 
number of people-related issues as well. First, applying M&S to the acquisition process 
requires properly educating and training acquisition personne! involved in the process. 
Program managers’ current lack of understanding regarding the implications of M&S 
represents a significant obstacle to effective use of M&S. The lack of a clear vision 
supported by achievable objectives for applying M&S contributes to the problem. 
However, the recent focus of Defense acquisition officials on utilizing M&S technology 
combined with formal training and education will hopefully improve M&S expertise 
throughout the acquisition community. 
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Acquisition managers indicated that resistance still exists to the use of 
constructive and virtual M&S, especially in the OT&E environment. Acquisition 
personnel must feel confident that models and simulations will produce credible data and 
results. Scarcity of convincing evidence showing M&S as a viable alternative to actual 
testing, and the lack of motivation to adopt M&S for conducting evaluations, contributes 
to this lack of confidence. As discussed earlier, DMSO and TRAC serve as the experts 
for implementing sound VV&A practices that may assist in addressing M&S credibility. 

The current stove-piped approach to weapon system acquisition does not 
efficiently produce a weapon system. Acquisition managers must apply the IPPD 
environment to weapon system development to facilitate the integration of M&S across 
functional disciplines. The IPPD environment better supports SBA through more 
effective communications and decision-making across all functional disciplines. 
Application of the IPPD environment will enable SBA to take advantage of the 
integration of functional disciplines across acquisition phases. These functional 
disciplines include contract management, test and evaluation, financial management, 
manufacturing, and logistics support. 

G. APPLICATION OF SBA TO THE ACQUISITION PROCESS 

From a theoretical standpoint, SBA can provide a new, more powerful acquisition 
Strategy potentially leading to significant product improvements. However, this assumes 
a common, reusable simulation-based acquisition environment that interfaces across all 
functional disciplines. This SBA environment will replace the traditional disjointed, 


stove-piped acquisition methodology with a collaborative IPPD environment that relies 
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on an iterative and interactive acquisition methodology for acquiring weapon systems. 


This concept envisions acquisition functional domains concurrently applying M&S 


throughout the acquisition process as depicted in Figure 9. 


To illustrate the strategy, assume the intelligence community proposes a new 


mission requirement reflecting improved threat capabilities. Articulating this requirement 


provides input parameters to models and simulations for defining and generating system 


operational performance requirements. 


Based on the new mission need, organizations 


such as TRADOC define performance requirements that provide a basis for developing 
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Figure 9. The SBA Environment (Frost, 1996). 
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and evaluating system operational and performance concepts. 

During Phase O of the new acquisition process, system designers and engineers 
will electronically access performance parameters stored in an information repository to 
explore various design concepts. Engineers create virtual designs using CAD tools that 
will also reside in the information repository. Other functional discipline experts can 
analyze and comment on the utility of each design on-the-fly in near real-time. Program 
managers study data collected from initial computer design and testing to begin 
addressing logistics support, financial management, and manufacturing concerns. 
Program managers feed this analysis back into the design process to determine which 
concepts best meet system performance requirements. As system and subsystem designs 
mature, program managers conduct tradeoff analysis of cost, schedule, and performance 
for each design alternative. 

By Phase J, the information repository contains information on operational 
scenarios for system testing, performance and design requirements, preliminary design 
solutions and associated system performance characteristics, material costs, and 
preliminary manufacturing requirements. The information repository provides program 
managers with information they need to make final design decisions based on analysis of 
cost, schedule and performance. In Phase I, system designers utilize engineering models 
to represent the weapon system in a three-dimensional, virtual environment. This gives 
the weapon system end-user the ability to visualize the new system and provide valuable 
feedback on system design. Virtual prototyping, combined with constructive simulations, 


will provide key insights into design modifications impact on system performance. 
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Significant cost savings may be realized through the use of M&S for testing at this stage 
rather than many iterations of live developmental tests. 

In Phase Il, SBA models and simulations link design and manufacturing processes 
and schedules. Contractors can utilize computer-aided manufacturing (CAM) methods to 
produce precise component prototypes and establish quality assurance guidelines for 
Statistical Process Control (SPC). Also during this phase, M&S supports configuration 
management through documentation of design changes leading to the final design. SBA 
makes it much easier to record and update the information repository for such things as 
design changes, system cost analysis, and performance evaluations. Through the 
information repository, this information becomes instantly available to other program 
managers, system users, and commercial contractors. 

Operational testing involves evaluating the system in a either a live, virtual, or 
constructive simulation environment. Data gathered from virtual testing provides 
engineers with insights into performance characteristics and is useful for making design 
changes. Within a mature SBA environment, conducting operational testing in a virtual 
environment can be accomplished at a much lower cost compared to live operational 
testing. 

During Phase III, results from field testing of system prototypes are added to the 
information repository along with final blueprints of the weapon system, and detailed 
documentation of the complete acquisition process. This information may be shared for 


future use. If the weapon system requires major modifications, program managers could 
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access historical system data when conducting future cost and performance evaluations of 
proposed changes, making design modification decisions, and production-line restarts. 

Throughout all acquisition phases, SBA will provide a system development 
environment for more effectively managing cost, performance, and schedule. This 
environment will be distributed and decentralized, not accomplished just from one 
location. Finally, it will provide program managers with access to use and reuse M&S 
data and products. This will reduce or eliminate costs associated with developing future 
models and simulations for specific applications. 
H. APPLICATIONS OF SBA-LIKE ENVIRONMENTS 

Examples of industry-related SBA ventures made thus far include International 
Business Machines’ (IBM) development of the Computer-Aided, Three-Dimensional 
Interactive Application (CATIA) to assist companies in utilizing the digital environment 
to efficiently produce quality products. As depicted in Figure 10, CATIA utilizes a 
centralized information database and consists of modular computer applications that form 
an integrated design, analysis, and manufacturing system. Users enter market data, field 
services (logistic support), and supplier’s parts or components into the database. 
Engineers use database information to create virtual designs. Analysis of designs may 
yield insights into cost, schedule, and performance to improve selection of an optimal and 


cost-effective solution. 
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Figure 10. IBM's CATIA Software (Garcia, 1993). 





CATIA database information can be accessed from distributed work-stations. 
Engineers, manufacturers, users, suppliers, financial managers, and analysts can all access 
information and make-decisions based on the most current design. Manufacturers can use 
final digital designs to develop unique tools and machines required for manufacturing and 
fabricating parts and components, assembly machines, and to define quality control. 

The Defense Advanced Research Projects Agency (DARPA) also conducted a 
SBA-like project to develop and validate a reusable, distributive, and collaborative 
process for designing weapon systems. The intent of the project was to develop and 
demonstrate a reusable framework that employs the IPPD environment, common 
applications, decentralized communication, and virtual prototypes in_ synthetic 


environments for the design, test, and fabrication of a product. 
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Lockheed Martin, selected by DARPA to lead development, successfully linked 
multiple tools across seven major areas: mission analysis, propulsion plant selection, 
collaborative design, distributive interactive simulation interactions, multi-disciplinary 
analysis, manufacturing analysis, cost analysis, and risk analysis (NATIBO, 1996). 
Lockheed validated the framework it developed) utilizing a notional, advanced-surface- 
combatant ship concept. Figure 11 overviews the project architecture. 

Much of DoD’s SBA framework discussed previously in this chapter came from 
the commercial sector’s efforts to apply M&S to the acquisition process. One noteworthy 
commercial SBA venture was Boeing’s design and development of the 777 passenger 
aircraft as depicted in Figure 12. This venture exemplifies how the SBA approach 


reduced both cost and development schedule, and improved performance. 
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Figure 11. SBD Architecture (Jones, 1997). 
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Figure 12. Boeing 777 Aircraft (Boeing, 1998). 


Boeing produced the first digitally-designed commercial passenger aircraft 
through the use of CATIA’s synthetic environment. CATIA allowed Boeing to fully 
design the complete aircraft on a computer. In one instance, Boeing capitalized on the 
CATIA environment by utilizing a synthetic mechanic to run simulations on the 
maintainability of the aircraft. Boeing found the computer-generated mechanic could not 
reach a particular navigation light in order to change the light bulb. By making necessary 
changes before ever building a single aircraft, engineers fixed this problem. Boeing also 
utilized the synthetic environment to check for interference between all parts of the 
aircraft. In one case this led to a decrease in assembly time from nineteen weeks to just 
three weeks. 

The CATIA application fostered the SBA concept of integrating functional 
disciplines across program phases. Through the use of design-build teams, Boeing not 
only forced design and manufacturing engineers to work side-by-side in the digital 


environment, but it also required suppliers and customers to participate throughout the 
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enuire process. Compared to producing other aircraft, Boeing reduced error and rework 
by 50%, delivered the first aircraft ahead of schedule, and kept total cost within budget 
(Marvil, 1998). 

The Grizzly represents one of the Army’s first attempts to utilize SBA in an 
ACAT II program. The Grizzly, built upon an M1 tank platform, is an obstacle breaching 
vehicle that features an automatic, depth-controlled, articulated mine clearing blade, and a 
telescoping excavating arm. Figure 13 depicts the system prototype. 

The program originally applied M&S in a disjointed and limited fashion. The 
program manager, Lieutenant Colonel (LTC) Kotchman, recognized the benefits of SBA 
and applied them, where possible, to the Grizzly acquisition process. LTC Kotchman 
assumed control of the program in Phase II of the acquisition process. He immediately 
focused on solving critical technical problems associated with the automatic depth 


control, systems integration, and man-machine interface (Kotchman, 1998). 








Figure 13. The Grizzley. 
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Through the use of models and simulations, the program manager evaluated 
characteristics of different hydraulic, suspension, chassis, and blade designs in simulated 
terrain. The Grizzly program interactively linked, for the first ttme, component level 
M&S products including a Computer Aided Project Engineering (CAPE) model, a 
propulsion, track, and suspension model called Dynamic Analysis and Design System 
(DADS), and Davis, a United Defense Limited Partnership proprietary terrain-animation 
model. The Grizzly program benefited from the application of interactive M&S by 
reducing a four-month, three-million dollar soils plowing study, to a one-day, three- 
hundred-thirty thousand dollar study. In another instance, the program utilized M&S to 
simulate components to reduce production interference errors during manufacturing. 
According to LTC Kotchman, the Simulation Support Plan (SSP) and M&S removed the 
‘fog of uncertainty” surrounding many technical and operational aspects of the process 
ultimately leading to reductions in cost and schedule, and performance enhancements 
(Kotchman, 1998). 

i; SBA RISKS AND BARRIERS 

Thus far, this thesis discussed benefits associated with implementing a simulation- 
based acquisition strategy. Acquisition program managers must also consider possible 
risks to implementing SBA. First, program managers cannot rely on constructive and 
virtual M&S for all acquisition related tasks. For example, DoD cannot rely exclusively 
on modeling and simulation for OT&E (DoD Regulation 5000.2-R, 1998). Program 
managers must fully understand current and applicable policies and laws that govern the 


use of models and simulations. 
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Second, data descriptions of some fielded equipment are not available. In other 
Cases, Managers have non-digital data descriptions. Program managers may find the cost 
of generating data descriptions or converting non-digital descriptions for legacy systems 
excessive. Further, utilizing unverified and invalid data descriptions could result in 
decreased system performance and increased costs. 

Third, program managers may face challenges in providing security for M&S 
technologies and resulting data. Some contractors retain proprietary control of computer 
models and simulations that they develop. This may cause configuration problems for re- 
use as well as associated costs. Also, since program managers conduct SBA in a 
distributive fashion, breaches of the information repository pose a security threat that 
program management personnel must address. 

Fourth, some software programmers did not base their model or simulation on 
physical properties. Instead, they based them on evidence gathered from other sources. 
This may create a situation where the model or simulation does not correctly interpret 
user-entered data, thus producing questionable or unreliable outputs. Movement towards 
more physics based models and simulations, rather than those based on historical data, 
will help provide greater acceptance in the acquisition community. 

Se SUMMARY 

The acquisition community only recently began to realize the potential impacts of 
SBA on the acquisition process. Continued improvements in technology and the 
appropriate application of M&S in an IPPD environment will solidify SBA as a method 


to develop and deploy complex weapons systems. Once fully developed, SBA will 
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provide the acquisition community with a new paradigm for significantly reducing both 
the acquisition cycle and costs while helping to develop a higher-quality. higher- 
performing product. However, these benefits also incur risks: lack of funds to fully 
develop and utilize M&S tools, inaccurate data descriptions, improper application of 
M&S tools, and security breaches of information repositories. The acquisition 
community must carefully consider both benefits and risks when developing a strategy for 


implementing SBA. 
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IV. SIMULATION-BASED ACQUISITON RESEARCH LABORATORY 
A. INTRODUCTION 
Chapter III discussed technological, cultural, and procedural barriers to 
implementing SBA. DoD’s focus on acquisition reform and sponsored research into 
modeling and simulation, computer, and communication technologies will contribute to 
overcoming barriers and accelerate the adoption of SBA as a strategy for weapon system 
acquisition. Another important area where the acquisition community has not yet fully 
realized the potential benefits of a SBA strategy is through the development of a 
simulation-based acquisition research laboratory. This type of laboratory can assist DoD 
and program managers’ efforts to exploit models and simulation for weapon system 
acquisition experimentation and program management. 
B. DEFENSE RESEARCH LABORATORIES 
DoD sponsors research to assist in technically-challenging, problem solving. 
Through DARPA, DoD spends over two-billion dollars annually to develop projects that 
will support current and future military needs. Further, the Army, Navy, and Air Force 
operate research laboratories to study technological areas that are relevant to their 
particular missions. For instance, the Air Force utilizes four laboratories: 
¢ Armstrong Laboratory, located at Brooks Air Force Base, Texas, focuses on 
human-machine integration problems. 
e Phillips Laboratory, located at Kirtland Air Force Base, New Mexico, deals 
with space and missile related research including geophysics, propulsion, and 


direct-energy weapons. 
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e Rome Laboratory, located at Griffis Air Force Base, New York, specializes in 
research and development of command, control, communications, and 
intelligence (CD) programs. 

e Wright Laboratory, located at Wright-Patterson Air Force Base, Ohio, 
conducts research and development of aeronautical technologies. 

These laboratories conduct cutting-edge research leading to development of new, state- 
of-the-art technology for the Air Force, contributing to development of new products and 
processes and improving mission success. Other laboratories include the Army Research 
Laboratory (ARL), Construction Research and Engineering Laboratory (CERL), Naval 
Aerospace Medical Research Laboratory (NAMRL), and Biomedical Research & 
Development Laboratory. 

Although each of these laboratories has a specific technological orientation, they 
all accomplish a common mission. Key objectives include the following: Discover 
enabling technologies that offer potential for revolutionary improvements in weapon 
systems; Develop and demonstrate advanced technologies for current and future 
requirements; Transition proven technologies to program managers; and Solve pressing 
technological problems. 

DoD research laboratories also form partnerships with academia and industry to 
assist In meeting these objectives. For example, ARL funded the University of 
Pennsylvania to develop a human performance model called Jack. The National 
Aeronautical and Space Administration (NASA) and the Tank Automotive Research, 


Development, and Engineering Center (TARDEC) both integrated Jack into their CAD 
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systems to research man-machine compatibility design issues (ARL, 1997). ARL and 
other laboratories also fund M&S technology and interoperability research. Although 
many research and development efforts support specific technical applications of 
technology, no strategic and holistic research program exists for SBA. 
c: SIMULATION-BASED ACQUISITION RESEARCH LABORATORY 
The SBA research laboratory environment may provide a concerted means to 
improve development and implementation of SBA. A simulation-based acquisition 
research laboratory would provide a proving-ground to develop, validate and implement 
the SBA process. Key research to be exploited by a SBA research laboratory are 
discussed below. 
e Discovery. This involves exploiting new M&S, computing, and 
communications technologies. Laboratory research would develop and test 
technologies that could assist in refining the SBA strategy to assist program 
managers in acquiring weapon systems more efficiently. The laboratory would 
capitalize on an advanced engineering environment to facilitate M&S integration 
and re-use across functional disciplines and programs for all Services. 
e M&S Development. Demonstrating SBA concepts using state-of-the-art M&S 
technologies to improve acquisition processes, the laboratory would simulate the 
acquisition process across functional disciplines to virtually design, analyze, and 
test system concepts. 
e Technology Integration. As the laboratory validates the SBA process and 


products, M&S components could then be exported to acquisition programs. 


36 


Laboratory modularity will allow reconfiguration and integration of new 

technologies as they become available for specific adaptation to unique program 

requirements. 

e Problem Solving. Applying new SBA technologies to improve business 

processes will create new problems. By using the laboratory to test new 

technology and integrate them with other components of SBA, the laboratory will 

not only solve complex technical problems but also address practical problems 

that occur when conducting acquisition across all functional disciplines and 

phases. 

With these four objectives in mind, the SBA research laboratory would provide 
insights into six key domains described below and depicted in Figure 14: 

e Command, Control, Communication, Computer, Intelligence, and 

Reconnaissance (C4SIR). Discover, develop and_ transition state-of-the-art 

technologies to assist program managers 1n utilizing high-speed, networked, data 

environments. 

e Acquisition Methodology. Evaluate M&S impacts on acquisition 

methodologies and propose new, more efficient methodologies for weapon 

systems procurement. 

e Prototyping and Manufacturing. Research and develop M&S products that 

will further exploit virtual prototyping and manufacturing of weapon systems. 

e Knowledge Mall. Develop and provide a one-stop, virtual “shopping mall” 


where users could access and utilize stored SBA or M&S related information such 
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Figure 14. SBA Research Laboratory Domains. 

as reusable data descriptions, models, simulations, and other pertinent acquisition 

information. 

e Knowledge Engineering. Discover, develop, and transition M&S products 

that support the functional analysis of the proposed weapon system. 

e = Virtual Environment. Discover, develop, and transition collaborative products 

to support the visualization of the weapon system focusing on emerging virtual 

reality technologies. 

In addition to providing research into these domains, the research laboratory 
would also provide a capability to initially test the SBA environment and assist the 
acquisition community in developing, designing, fielding, managing weapon systems. 
The SBA research laboratory would support defense acquisition through the “top-gun” 
approach by concurrently researching, developing, and testing SBA focusing on a 
federation of applications, high-speed distributive communications, collaborative 


environments, and IPPD. 


a) 


Figure 15 depicts the research laboratory which, through collaboration with 
academia and industry, pursues and applies the necessary standards and protocols, 
architecture, infrastructure and digital descriptions required to create the federation of 
applications. This federation of applications supports seamless and virtual development, 
analysis, and production of weapon systems. Once acquisition personnel create the 
weapon system, the digital blueprint becomes part of the knowledge mall. Additionally, 
as M&S applications are utilized for running distributive training simulators, performing 


operational analysis, and analyzing future system upgrades, resultant data would be fed 
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Figure 15. Laboratory Environment. 
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back into the knowledge mall. Development of the federation of applications provides a 
virtual environment with necessary interactivity to seamlessly support all phases of the 
acquisition process as well as soldier and crew training, operations analysis, and system 
upgrades. 

Figure 16 depicts a high-level architecture for the SBA research laboratory. This 
architecture reflects a laboratory environment for not only conducting research and 
development into each of the domain areas but also conducting proof-of-principle 
demonstrations. To provide the SBA capability, the laboratory would consist of 
computer stations and high-fidelity, graphic-visualization displays that will provide 
information to acquisition functional areas such as test and evaluation or logistics 
analysis. Networked computers would provide varying levels of fidelity based on user 
needs. 

Networked computers also facilitate creation of a “knowledge mall.” In concept, 
the knowledge mall would provide one-stop access to stored information such as data 
descriptions, terrain data, and models and simulations necessary to conduct SBA. 
Capitalizing on state-of-the-art computing and communication technology, this SBA 
proving-ground could either be centralized or decentralized. Further, other organizations 
such as training units, battle labs, and acquisition programs, could connect to the network 
from distant sites to participate in ongoing SBA activities or access specific products in 


the resource repository. 
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Figure 16. SBA Research Laboratory Architecture. 


Figure 17 diagrams the laboratory process. The laboratory environment consists 
of all items in the shaded region of Figure 17. Products and interfaces with other 
agencies are depicted outside of the shaded area. The laboratory environment interfaces 
with industry and academia to investigate state-of-the-art technologies, acquisition and 
engineering knowledge, and improving processes that contribute to the creation and 


success of SBA. The knowledge mall, computer network, and visualization capability 
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Figure 17. SBA Laboratory Process. 


combine to create the enabling environment. Within this environment, managers could 
conduct simulation-based acquisition. 

From a process perspective, the requirements community feeds mission needs into 
the laboratory. Laboratory personnel conduct virtual systems engineering, develop 
performance requirements, and design concepts to meet DoD needs. Personnel also 
conduct functional analysis of concept designs to assist in eliminating designs or design 
elements that do not meet specific performance objectives. 

Functional disciplines, such as financial management and logistics support, also 
represent potential research areas for conducting risk analysis and mitigation as well as 


trade-off studies. This analysis helps narrow design concepts and alternatives to ones that 
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best meet cost, performance, and schedule requirements. Researchers conduct simulated 
test and evaluation on completed designs, verifying that the virtual system meets required 
performance, operational effectiveness, and operational suitability thresholds. A modular 
design will enable the laboratory to incorporate emerging virtual reality technology. This 
would further enhance the ability of the laboratory to measure design suitability and 
resolve man-machine interface problems. Digital system design research would provide 
opportunities to model computer-aided manufacturing processes. It may also be possible 
to link industry with the laboratory environment to capture electronic designs necessary to 
produce parts and components for the complete system. 

Through the SBA research laboratory, it will be possible to design a new weapon 
system in a virtual environment and also develop new business processes. New business 
practices, derived from lessons learned in conducting IPPD as part of the SBA 
environment, would provide greater efficiency to the acquisition process. The laboratory 
would then feed these new business processes back to the acquisition community and the 
enabling environment for further refinement of the acquisition process. 

D. SUMMARY 

The SBA research laboratory concept is an important component of SBA 
integrating best M&S technologies and practices across all services, industry and 
academia. It would assist the acquisition community in implementing the SBA strategy 
while providing a virtual proving-ground for conducting SBA prototyping. The SBA 
research laboratory represents a centralized research facility for discovering, developing, 


transitioning and solving pressing SBA related technological and process issues. Finally, 
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it provides an environment to accelerate the development, adoption, and implementation 


of SBA. 
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V. CONCLUSION 
A. OVERVIEW 

Traditional Defense acquisition methodology, combined with declining defense 
budgets and increasing weapon system complexity, created a resource intensive 
acquisition process. To combat this problem, DoD implemented acquisition reforms, 
such as the IPPD environment, to reduce resources necessary to acquire weapon systems. 
DoD recognized that computer models and simulations saved or mitigated costs, assisted 
in reducing cycle-time, and helped improved product performance. However, DoD did 
not fully appreciate the benefits of a synchronized M&S environment for acquiring 
weapon systems. 

Historically, program managers applied modeling and simulation in a disjointed, 
stove-piped fashion. Acquisition programs developed computer models and for specific 
functional uses and often did not consider interface or re-use applications. Given 
advances in computing technologies, communication networks, and Defense 
methodologies for weapon system acquisition, a new paradigm emerged, that of 
simulation-based acquisition. 

SBA capitalizes on the IPPD culture, a spiral or evolutionary methodology, and an 
advanced technological environment. It provides modeling and simulation assets 
necessary to support the concept development, risk reduction, engineering, and 
manufacturing of a system across all functional disciplines. SBA accomplishes this 


through integrated and distributive technologies rooted in the IPPD environment. SBA 
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will provide the acquisition community with the strategy to acquire weapon systems 
faster, cheaper, and with higher performance. 
B. BENEFITS 

The traditional acquisition methodology does not support the SBA concept. The 
highly linear nature of this methodology does not facilitate the integrated, cross- 
functional approach required by the virtual environment. SBA functionality reflects a 
process that compresses the acquisition cycle through the use of IPPD and a model-test- 
model approach. More useful methodologies are either the evolutionary or spiral 
processes, where iterative applications of systems engineering processes occur over time 
until the most cost-effective and operationally-effective concept emerges. 

Program managers often developed and utilized unique modeling and simulation 
applications that did not support re-use across phases, disciplines, or programs. By 
creating a federation of interactive computer models capitalizing on advancements in 
computer and communication technologies, a distributive and collaborative virtual 
environment can be created. This SBA environment would provide necessary 
applications to conduct near simultaneous design, analysis, and testing of a concept 
system across all functional disciplines. 

Following a SBA approach to weapon system acquisition presents program 
managers with both risks and benefits that must be addressed. Nonetheless, the benefits 
seem to greatly outweigh the risks. Through application of computer models and 


simulations, program managers will reduce cycle-time and costs, and improve system 
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performance. Reliance on the IPPD process will foster greater cooperation among all 
stakeholders to produce a more effective system. 

Results suggest SBA has the potential to reduce system acquisition cycle-time by 
fifty percent, lower costs, and increase system performance. However, an integrated SBA 
environment is not currently available. Therefore, DoD develop an integrated computing, 
software, communications, and business environment to fully realize the potential of 
SBA. 

To this end, a SBA research laboratory will assist DoD and program managers 
with facilitating development and implementation of SBA. Through centralized efforts to 
synchronize SBA elements across all Services, the research laboratory could accelerate 
the development and implementation of SBA. The laboratory could provide technical 
assistance, training, and education opportunities to program offices that wish to utilize 
M&s products or apply SBA strategies. The laboratory also serves as a research center 
for virtual environments, evaluating combinations of technologies and business practices. 
c: RECOMMENDATIONS 

SBA will provide the program manager with a viable and potentially powerful 
Strategy to mitigate risks and render a more timely, cost-effective weapon system. As 
SBA continues to evolve and mature with improvements in computers, information 
technology, and software, Defense acquisition officials must continue to facilitate this 
maturation by providing the leadership and resources to develop and define the enabling 


architecture required to fully integrate different models and simulations. 
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e DoD must incentivize M&S investment. Incentives will motivate program 
managers to explore and implement models and simulations when appropriate. 

e DMSO must continue to serve as the focal point for M&S technology 
management. It should synchronize SBA efforts across all Services to assist in 
removing redundancy and accelerating the development of this acquisition 
Strategy. 

e DMSO should create a SBA research laboratory environment. This laboratory 
would assist in synchronizing and coordinating M&S related research efforts to 
produce the SBA environment. It would accelerate the adoption and 
implementation of a SBA strategy and provide the acquisition community with a 
focal point for this new concept. 

e The acquisition community must continue to mandate the use of M&S when 
appropriate. Education and training of acquisition professionals, testers and 
evaluators, and users alike will help make M&S a more accepted and understood 
tool and overcome biases toward M&S. 

e Program mangers should develop a strategy early on to incorporate the 
interactive use of M&S technologies across all phases and functional disciplines. 
The SSP represents a critical document for communicating this strategy. 

e Program managers must implement the IPPD environment to fully realize the 
potential of SBA. The program manager must involve all stakeholders including, 


users, testers, and suppliers in the acquisition process as early as possible. 
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e Program mangers must secure M&S program funding. The SSP will assist in 
defining the required models and simulations and projecting the associated costs 
of obtaining and operating these programs. 
D. AREAS FOR FURTHER RESEARCH 
The following areas may provide additional research opportunities for SBA 
related topics: 
e Further investigation of current progress toward implementing the 
technologies required to create the SBA environment is warranted. Since the 
elements of SBA are evolving at a rapid rate, further research would provide new 
insights into areas that challenge the implementation of SBA. Results of this 
research could suggest methods for overcoming these problems. 
e Compare a traditionally developed weapon system with one developed using 
the SBA strategy. This research would provide a detailed cost-benefit analysis 
between the two approaches to quantify the actual benefits of SBA, and discuss 
lessons learned from the implementation of SBA. 
e Further development of a SBA research laboratory architecture may prove 
useful to DoD. Research the development of lower-level functional requirements 
and interfaces to provide a more detailed laboratory architecture. Additional 
research into available, enabling technologies and their integration into the 
laboratory would provide DoD with a blueprint for moving from concept to 


reality. 
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APPENDIX 


AE Acquisition Executive 

AMSO Army Modeling and Simulation Office 

ARL Army Research Laboratory 

ATD Advanced Technology Demonstration 

AWE Army Warfighting Experiment 

C3i Command, Control, Communications, and Intelligence 

C4SIR Command, Control, Communication, Computer, Intelligence, and 
Reconnaissance 

CAD Computer Aided Design 

CAIV Cost as An Independent Variable 

CAM Computer Aided Manufacturing 

CAPE Computer Aided Project Engineering 

CASTFOREM Combined Arms Support Task Force Evaluation Model 

CATIA Computer-Aided, Three-Dimensional Interactive Application 

CERE Construction Engineering and Research Laboratory 

CINC Commander-in-Chief 

DADS Dynamic Analysis and Design System 

DARPA Defense Advanced Research Projects Agency 

DMSO Defense Modeling and Simulation Office 

DoD Department of Defense 

DSMC Defense Systems Management College 

DT&E Developmental Test and Evaluation 

EC Electronic Commerce 

EDI Electronic Data Interchange 

EMD Engineering and Manufacturing Development Phase 

FAR Federal Acquisition Regulation 

FASA Federal Acquisition Streamlining Act of 1994 

FCIM Flexible Computer-Aided Integrated Manufacturing 

GWEF Guided Weapons Facility 

HLA High-Level Architecture 

HW Hardware 
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IBM 
IEP D 
IPT 


JCS 
JROC 
JSF 
JTA 


LRIP 


M&s 
MAA 
MDA 
MDAP 
MNS 
MSRR 
MTS 


NAMRL 
NASA 
NATIBO 
NDIA 


OMB 
OT&E 


PDRR 
PM 
PMO 


REFORGER 


SBA 
SBD 


International Business Machines 
Integrated Product and Process Development 
Integrated Product Team 


Joint Chiefs of Staff 

Joint Requirements Oversight Council 
Joint Strike Fighter 

Joint Technical Architecture 


Low Rate Initial Production 


Modeling and Simulation 

Mission Area Analysis 

Milestone Decision Authority 

Major Defense Acquisition Program 

Mission Need Statement 

Modeling and Simulation Resource Repository 
Moving Target Simulator 


Naval Aerospace Medical Research Laboratory 

National Aeronautical and Space Administration 

North American Technology and Industrial Base Organization 
National Defense Industrial Association 


Office of Management and Budget 
Operational Test and Evaluation 


Program Definition and Risk Reduction 
Program Manager 
Program Management Office 


Return of Forces to Europe 


Simulation-Based Acquisition 
Simulation-Based Design 
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Se ® 
SSP. 
STEP 
SWIL 


TARDEC 
TACOM 
TRAC 
TRADOC 


USD(A&T) 


VIC 


VV&A 


WAAM 


Statistical Process Control 

Simulation Support Plan 

Simulation Test and Evaluation Process 
Software in the Loop 


Tank Automotive Research, Development, and Engineering Center 
Tank Automotive Command 

Training and Doctrine Analysis Center 

Training and Doctrine Command 


Under Secretary of Defense (Acquisition and Technology) 


Virtual Individual Combatant 


Verification, Validation, and Accreditation 


Wide-Area, Anti-Armor Munition 
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